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Abstract 

The entropy density is an intuitive and powerful concept to study the complicated nonlinear pro- 
cesses derived from physical systems. We develop the minimum entropy density method (MEDM) 
to detect the structure scale of a given time series, which is defined as the scale in which the un- 
certainty is minimized, hence the pattern is revealed most. The MEDM is applied to the financial 
time series of Standard and Poor's 500 index from February 1983 to April 2006. Then the temporal 
behavior of structure scale is obtained and analyzed in relation to the information delivery time 
and efficient market hypothesis. 
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I. INTRODUCTION 



In recent years, physicists have enlarged the research area to many interdisciplinary fields. 
Econophysics is one of the active research areas where many statistical methods are applied 
to investigate financial systems. Many analytic methods are introduced, such as the cor- 
relation function, multifractality, minimal spanning tree, and spin models [l|, 3, O, li, O, Q]. 
The empirical time series in financial markets have also been investigated by using vari- 
ous methods such as rescaled range (R/S) analysis to test the presence of correlations [71] 
and detrended fluctuation analysis to detect long-range correlations embedded in seemingly 
non-stationary time series j^l, 

In this paper we focus on how to find a specific time scale in which a pattern in a time 
series is revealed most. Since pattern can be interpreted as the repetitive structure inside the 
time series we will be referring to that specific scale as structure scale. To find this structure 
scale we introduce the minimum entropy density method, which will be elaborated in detail 
and exemplified with the cases of finite periodic time series with corruption in Section [Til It 
is because the periodic time series is simple and has a repetitive structure among it definitely. 
However, our method can be applied to the other time series as well as the other processes, 
such as configurations of spin chain, if they have any certain structures. As an example of 
empirical analysis we apply this method to the time series of S&P500 index in Section IIII1 
The temporal behavior of the structure scale of the index is obtained and the implications 
of the result is analyzed in relation to the information delivery time and efficient market 
hypothesis. 



II. MINIMUM ENTROPY DENSITY METHOD 



A. Backgrounds 

Since our new method for finding the structure scale of a finite time series is based on the 
information theory, we start with briefly explaining the concepts in the information theory 
according to Ref. [l0|]. Firstly, we consider a process given by an infinitely consecutive 
discrete random variables, X = ■ ■ ■ X^xXqX± X^ • ■ ■ , where each Xj may take the value Xi 
drawn from a finite countable set A of size k. The probability distribution of a block of L 
consecutive random variables X L = Xi, ■ ■ ■ , Xi + i-i is taken as the set of joint probabilities of 
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L consecutive values Pr(x L ) = Pr(xj, • • • , x i+ L-i) for all k L possibilities. Then the Shannon 
entropy for the above L-block variable X L is defined as 

H(L) = -J2---J2 Pr (zi," - ,x L )log 2 Pr(a;i,.-. ,x L ), (1) 
x 1 eA x L eA 

which measures the uncertainty or randomness in the process. H(L) is a monotonically 
increasing function of L because the more relevant information can be extracted from the 
time series for the larger L. We can measure the entropy of the infinite process X by 
taking L — > oo. However, H(L) may diverge as L goes to infinity so an entropy density is 
introduced as follows: 

K - lim SS, (2) 

equivalent ly 

h^= lim {H(L + 1) — H(L)}. (3) 

If the process X contains a periodic structure, for a sufficiently large L (larger than the 
period) increasing L does not give us any more information. In this case the entropy density 
becomes 0. On the other hand, if the process has been generated totally randomly, Pt(x l ) = 
k~ L for all k L possibilities, then H(L) = L log 2 k and consequently = log 2 k, which is 
the maximum value of the entropy density. Therefore the repetitive structure embedded 
in the process makes the entropy density lower than that of a more random process. In 
addition the entropy density can be interpreted as the uncertainty of a given variable when 
all the preceding variables are known. If there exists a repetitive structure in the process, 
the knowledge of all the previous information will greatly decrease the uncertainty of the 
next variable. 

Since the finite size of the empirical data sets directly a limit to the block size L, we need 
the finite-L approximation to the thermodynamic entropy density as follows: 

h^(L) = H(L)-H(L-l), L = l,2,..., (4) 

where H(0) is set to 0. Actually all the processes we deal with through this paper are 
finite, hence only h^{L) matters other than h^. By the way, unless L is large enough to 
fully detect the structure in the process, h^(L) would overestimate the randomness of the 
process. Therefore, as L increases h^(L) converges to h^. 
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B. Method 



For the entropy density analysis we coarse-grain the data with an appropriate scale and 
then digitize the continuous amplitudes into discrete values. Let us consider a temporal data 
set Y(t) as a function of discrete time steps t — 0, • • • ,S — 1. Once the scale s to grain the 
data is given, then the resulting time series has N = S/s equally spaced measurements. For 
the digitization we set a countable set A to the smallest and simplest set of size k = 2, such 
as {0, 1}, among the various alternatives. In other words the original data set Y(t) changes 
into the binary time series F n by the following process: 

F n = 9(Y(sn + s)-Y(sn)), < n < N - 2, (5) 

where 9(x) is a Heaviside step function. F n gets the value of if the value of measurement 
has decreased after the interval s and does the value of 1 otherwise. To make clear the effect 
of choosing s on the coarse-grained data set, consequently on the entropy density we define 
h^(s, L) as the entropy density of the process coarse-grained with scale s, which plays a key 
role in our method. 

The minimum entropy density method (MEDM) is based on the assumption that the 
pattern in the time series is revealed most when the time series is coarse-grained with the 
structure scale defined as the scale minimizing the entropy density. Most empirical time 
series for the complex systems are usually contaminated by the high frequency noise and 
we want to get the noiseless signal or intrinsic structure from the scratches. Provided that 
a time series can be described by a characteristic time scale s c , if the smaller scale than s c 
is used for the analysis the time series looks more random due to the high frequency noise. 
On the other hand if the larger scale than s c is used we overlook the time series so that we 
fail to get the structure, hence the time series looks more random too. In short, by finding 
the scale minimizing the entropy density we can get the characteristic scale s c . 

For the first step of MEDM we decide the range of coarse-graining scale, usually set to 
[l,s m0 x]- Then, before finding the structure scale s* minimizing h^(s,L) by tuning s we 
should determine the appropriate value of L. The first of two criteria for choosing L sets 
the upper bound of L: 

L < log, N, (6) 
where N is the number of data points S/s and k is the size of the set A. As L increases, 
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the more relevant information can be extracted from the process while the average number 
of realizations for each possibility of L-block variable decreases fast as N/ k L for finite N. 
Therefore, for the significant analysis L should be limited by a condition that the average 
number of realizations for each possibility of L-block variable should be at least one: N[k L > 



1, equivalent to Eq. (jBD. For the mathematically rigorous arguments see Refs. [12l. Il3|. 

The second criterion is to determine the convergence range of L in which /i M (s, L) for 
some s converges to h^. However, without the knowledge of it is not clear to see whether 
h^(s, L) converges to or not. If the value of s* does not depend on L we can determine 
the structure scale s* even though /i M (s, L) does not converge yet. On the other hand, if the 
value of s* varies according to L in general we have to find the convergence range of L, which 
can be practically defined as the range where the landscape of h^s, L) is approximately flat. 
Once such a convergence range exists for any s, it would be enough to determine s* for that 
range of L because the aim of this paper is to compare the entropy densities for different 
scales for a fixed value of L, not to get the better approximation of entropy densities. 

The MEDM that we have discussed so far can be summarized into four main steps 
including how to choose an appropriate value of L: 

1. Decide the range of coarse-graining scale, usually set to [1, s max ). 

2. For each s in that range, transform the given time series Y(t) into /c-ary time series, 
for example, by Eq. (JSJ). 

3. Choose the appropriate value of L*: 

(a) L should be lower than log fc N, where iV = S/s max . 

(b) L should be chosen inside the convergence range where the landscape of h^s, L) 
is flat for the whole range of s. 

4. Find the structure scale s* minimizing the entropy density h^s, L*) by tuning s. 

For the last step of MEDM there may be more than one minimum in the landscape of 
hfj,(s,L*), which will be discussed with the examples in the next Subsection. 
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C. Model examples 



The MEDM is applied to the finite periodic time series with corruption because the 
periodic time series is simple and has a repetitive structure among it definitely. We consider 
the following series: for each time step t, 



where p is the period of Y(t), rj is a random number uniformly drawn from [0, 1], and r 
represents the fraction of corrupted data points. We set the data size S to 10 5 , p to 50, k 
to 2, and s max to 200, i.e. the 200 different sets of binary time series are constructed by 
Eq. (jHJ). Then for a few cases of r the entropy densities for the whole range of s and for 



When there is no corrupted data, i.e. r = 0, for L < 3 the global minima of /^(s, L) 
turn out to be for the multiples of s = 25 = p/2. As I increases there appears the 
additional global minima for the other values of s. Finally the entropy densities h^s, L) for 
the multiples of s = 5 become the global minima when L = 5 and even when L > 5. This 
implies that L > 5 is the convergence range of L as shown in Fig. [2J One can criticize that 
for the case of s = 1, where the binary time series becomes F = o 25 l 25 25 l 25 • ■ ■ , the pattern 
25 1 25 can be completely revealed by taking L larger than 50. But it is contradictory to the 
first criterion in Eq. ([6]) when given the finite time series. Instead of taking L as large as 
possible, we can get more relevant results by tuning the scale s even for the small values of 
L guaranteeing the significance of analysis. 

If the corruption is taken into account, i.e. in cases of r = 0.1 and 0.5, the local minima 
for the odd multiples of s = p/2 become distinctive among other minima (Fig. [1] (b) and 
(c)). Moreover, the values of distinctive local minima in the landscape of h^s, L) turn out 
to be independent of L so that the structure scales s* are successfully determined and hence 
it is not necessary to specify the convergence range of L as well as L*. 

Then why do the entropy densities for the odd multiples of s = p/2 remain minimized? 
The corruption definitely destroys the periodicity of time series and increases the random- 
ness, therefore the overall values of local minima of L) get larger than those for the 
case of r = 0. However, the effect of corruption is not uniform. At first, without corruption 
Y(sn) in Eq. ([5]) for the odd multiples of s = p/2 take the extreme values of Y(t), precisely 




with probability 1 — r 



with probabihtyr 



(7) 



L < 8 < log 2 iS^ are calculated, as partly shown in Fig. [U 
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Y(pn/2) = 1 for n even and Y(pn/2) = for n odd. Therefore the flip probability, defined 
as the probability that the sign of argument Y(sn + s) — Y(sn) in Eq. ([5]) is flipped due 
to the corruption, is the least among for the other values of s. For example, if Y(pn/2) 
remains unchanged as either or 1 while Y(pn/2 + p/2) is replaced by a random number in 
[0, 1], the flip probability is 0. On the other hand, for the case of s = p, if Y(pn) remains 
unchanged as 1 while Y(pn+p) is replaced by a random number in [0, 1], the flip probability 
is 1. As a result one can expect that when the periodic function is corrupted by noise, the 
most robust scale is not the period p and its multiples but p/2 and its odd multiples. Based 
on this argument one can say that the existence of more than one distinctive local minimum 
naturally comes from the repetitive structure of the original function Y(t), and that the 
patterns can appear in the different scales simultaneously. 

For more general application to a continuous time series we can take the finer scales to 
increase the precision of measuring the structure scale. To show an efficient way to fine- 
tune s we consider a corrupted periodic function with non-integer period, for example, for 
a continuous time t, 



To measure the structure scales (the odd multiples of s = p/2 for the case of discrete periodic 
functions), s should be smaller than 0.1. Instead of scanning the whole range of s, such as 
from 0.1 to 100.0, by the increment of 0.1 we tune s in a larger scale first and then move 
down to the smaller scales. The value of L is set to 6 according to the MEDM. Figure [3]^a) 
shows the entropy densities for various s in the order of 10. The minimum of the entropy 
density occurs at s = 40. We narrow the variation of s down to 1 around 40. Then the 
minimum of the entropy density occurs at s = 39 in Fig. [3]^b) and we repeat the same 
process again. Finally, in Fig. [3]^c) we obtain s* = 38.8 minimizing the entropy density, 
which is exactly a half period {p/2). 

Finally, the MEDM can be applied to a periodic function with varying period by dividing 
the given time series into several regions and applying MEDM to each of them. Here we 




with probability 0.9 
with probability 0.1 



(8) 
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consider a periodic function with linearly decreasing period: for a continuous time t, 

{cos 2 [ -TTvi ) with probability 0.9 
\p® J ( 9 ) 
T) with probability 0.1 

P(t) =Pi + ^^t, (10) 

where the period continuously decreases from p\ to p2- We set pi to 50, P2 to 40, and S to 
10 5 , respectively. The total time series is divided into 10 regions and the MEDM is applied 
to each of them. For all the regions we tune s in an order of 1 and set L to 6 after testing 
in a way we described before. Figure H] shows that the smallest structure scale s* decreases 
from 25 = pi/2 in the first region to 20 = P2/2 in the last one. These s*s are exactly the 
half periods of the starting and ending parts of the original function. If we divide the time 
series into more regions and use the finer scales, then the resultant temporal behavior of 
structure scale gets closer to p(t)/2, where p{t) is defined in Eq. ffTUj) . than before. 



III. EMPIRICAL DATA ANALYSIS 



Now we apply the MEDM to analyze the financial time series of the S&P500 index from 
year 1983 to 2006. We used the tick-by-tick data. It is reasonable to think that the structure 
scale of S&P500 index for 24 years would change from time to time. Hence the formalism of 
the last example in the previous Section is used. It should be noted that although the time 
series of the S&P500 index is not periodic, we can always measure the structure scale using 
MEDM whenever the series has patterns. 

The total time span of the index data from February 1983 to April 2006 is divided into 
279 regions, i.e. each region for each month. For each region the structure scale is obtained 
then the temporal behavior of it will be analyzed. The unit of coarse-graining scale s is set 
to 1 tick, the finest resolution of the empirical S&P500 index data. On average there are 
4 ticks in one minute though the real time intervals between adjacent ticks are not equally 
distributed. One reasonable way to fix this problem is to obtain the structure scale s^ ick in 
a unit of tick for each month and multiply it by the average real time interval fu c k between 
ticks within that month. The resulting value s* = s^ ick ■ f tick will be the structure scale in a 
unit of time for each month. 

Then we follow the four main steps of the MEDM to measure the structure scale of tick 
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every month. For the first step the range of Stick is set to 1 tick to 30 ticks. The tick series 
with suck = 30 has less than 900 data points each month. By Eq. (jSJ) the upper bound of L is 
9. Considering the second criterion of choosing L, we set L* to 5 by finding the convergence 
region of L for the whole range of s t i C k- Figure shows the landscapes of entropy densities 
hfj,(su c k, L) only for the regions of February 1983 and April 2006. For the third step the 
structure scale s* tick minimizing h^suck,^) is determined for each month. Three examples 
are shown in Fig. [61 where h^{s t i C ki 5) is minimized at s t i C k — 8 for January 1987, at s t i C k — 6 
for January 1996, and at s t i C k — 2 for January 2001, respectively. Unlike the case with the 
periodic time series, for each region there is only one structure scale over the range of s. 
After finding all the s^ ick we convert them into the real time scales by multiplying the average 
time interval between ticks for each month. Finally we get the temporal behavior of the 
structure length s* as shown in Fig. [7J During 1980's and 1990's s* decreases slowly but 
declines fast after late 1990's. 

We analyze the meaning of this result by considering the time scale by which the infor- 
mation flows among interacting agents in the stock market. The stock market price changes 
only when the agents in the stock market buy or sell. Since the agents make decisions based 
on the information they get, the information delivery time can be one of the most important 
factors for the changing rate of price. The information delivery time (IDT), defined as the 
time taken for the delivery of information from sources to agents, is assumed to be propor- 
tional to the average price change cycle. If the entropy density of the time series is measured 
with scale s smaller than the IDT, it would be relatively high because the coarse-grained 
time series looks more random due to the high frequency noise. On the other hand, if s 
is larger than IDT, we overlook the pattern embedded in the time series so fail to detect 
the structure scale and the coarse-grained time series looks more random too. Therefore, 
if the optimally closest scale to the IDT is used to detect the patterns in the time series, 
the entropy density for that scale would be minimized due to the repetitive structure of the 
price change. Consequently, 

IDT « s*. (11) 

The long-term decrease of s* from year 1986 to 2006 in Fig. [7J can be interpreted as the 
decrease of the information delivery time. The value of s* suddenly jumps down around 
year 1997, when the Internet was starting to spread widely, the fraction of online traders 
increased exponentially. These influenced the IDT of the stock market to become much 
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shorter. 

Since there does not exist any standardized way to measure the information delivery 
time, we suggest s* as one of candidates to measure it. IDT can be also used to measure 
the efficiency of the stock market: if the market is idealized with efficient market hypothesis 
(EMH) [2]], then IDT will become 0. In addition from our quantitative analysis IDT of the 
S&P500 index is about 17 seconds in year 2006. 



IV. CONCLUSIONS 



In this paper we have developed the minimum entropy density method (MEDM) to detect 
the structure scale of a given time series. This method is based on the assumption that the 
pattern in the time series is revealed most when the time series is coarse-grained with the 
structure scale defined as the scale minimizing the entropy density. We also showed that the 
MEDM is useful to detect the repetitive structures in the various time series if they have 
certain patterns. 

Additionally, by applying the MEDM to the financial time series of S&P500 index we 
identified that the time scale with the most patterns showing, has decreased for the last 
twenty years. In other words the information flows faster than before. The MEDM has 
also been applied to Korea Composite Stock Price Index (KOSPI) from April 1992 to June 
2003 with 1 minute time interval [14J]. The structure scale of the KOSPI index, which can be 
interpreted as the IDT, had also decreased for ten years similar to S&P500 index. We believe 
this effect is real, considering that the Internet trading has become popular recently, which 
we think is one of the main factors of decreasing the IDT, in both U.S. and Korean stock 
market. Also, Yang and colleagues [ijj] used the microscopic spin model to investigate the 
financial market and identified that the change of log-return distributions of financial stock 
markets can result from the increasing velocity of information flow, which implies that the 
IDT becomes shorter than before. Since IDT measures the efficiency of the stock market, 
by quantitative analysis we conclude that the efficiency of the U.S. stock market dynamics 
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became close to EMH. 
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FIG. 1: (Color online) The landscapes of entropy densities hn(s,L) of the periodic time series as 
functions of scale s with block size L = 2 (black circles), 4 (red crosses), and 6 (blue plus signs), 
respectively. The fraction of corrupted data points r is (a), 0.1 (b), and 0.5 (c), respectively. For 
(b) and (c) each point is averaged over 50 realizations and for a clear view we plotted s to 100 not 

to Srnax ~ 200. 
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FIG. 2: The entropy densities h^s, L) of the time series coarse-grained with scales 20 (plus signs), 
80 (crosses), 140 (circles), and 200 (squares) when the fraction of corrupted data points is 0.1. 
Each point is averaged over 50 realizations. There exists a convergence range of L in [5,8]. 
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FIG. 3: The entropy densities h^(s,L = 6) measured in the precision of 10 (a), 1 (b), and 0.1 (c), 
respectively. 
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FIG. 4: The temporal behavior of the structure scale s*, where each point represents the s* for 
each partitioned region. 
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FIG. 5: The entropy densities h^(s,L) of S&P500 index in February 1983 (a) and in April 2006 
(b) using the time series with scale stick = 1 (squares), 2 (triangles), 5 (circles), and 10 (plus signs), 
respectively. 
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FIG. 6: The entropy densities h fl (s, L = 5) of S&P500 index measured in January 1987 (squares), 
January 1996 (triangles), and January 2001 (circles), respectively. 
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FIG. 7: The temporal behavior of the structure scale s* of the S&P500 index measured monthly 
in a unit of time. 
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